Generalized Winner-Relaxing Kohonen Self-Organizing Feature Maps

We calculate analytically the magnification behaviour of a generalized family of self-organizing feature maps inspired by a variant introduced by Kohonen in 1991, denoted here as Winner Relaxing Kohonen algorithm, which is shown here to have a magnification exponent of 4/7. Motivated by the observation that a modification of the learning rule for the winner neuron influences the magnification law, we introduce a generalization entitled as Generalized Winner Relaxing Kohonen algorithm that reaches a magnification exponent of one and therefore provides optimal mapping in the sense of information theory. Between both variants a parameter allows to interpolate, and the unmodified Kohonen map appears as special case inbetween both variants. Compared to the original Self-Organizing Map and some other approaches, the generalized Winner Enforcing Algorithm requires minimal extra computations per learning step and is conveniently easy to implement. As the brain is assumed to be optimized by evolution for information processing, one would postulate that maximal mutual information is a sound principle governing the setup of neural structures. For feedforward neural structures with lateral inhibition, an algorithm of maximal mutual information has been defined by Linsker (Linsker 1989) using the gradient descend in mutual information. It requires computationally costly integrations, and has a highly nonlocal learning rule and therefore is less favourable as a model for biological maps and less feasible for technical applications. However, both biological network structures and technical applications are (due to realization constraints) not necessarily capable of reaching this optimum. This remains a question under discussion especially for the brain (Plumbley 1999). Even if one had quantitative experimental measurements of the magnification behaviour, the question from what self-organizing dynamics the neural structure emerged remains. So overall it is desirable to formulate other learning rules that minimize mutual information in a simpler way. The self-organizing feature map algorithm proposed by in 1982 by Kohonen (Kohonen 1982) has become a successful model for topology preserving primary sensory processing in the cortex (Obermayer et. al. 1992), and an useful tool in technical applications (Ritter et. al. 1992). Self Organizing Feature Maps map an input space, such as the retina or skin receptor fields, into a neural layer by feedforward structures with